Deep Learning for mapping retrogressive thaw slumps across the Arctic

Retrogressive thaw slumps (RTS) are typical landscape processes of thawing and degrading permafrost. To this point, their distribution and dynamics are almost completely undocumented across many regions in the permafrost domain, partially due to the lack of data and monitoring techniques in the past. We are tackling this shortcoming by creating a deep learning based semantic segmentation framework to detect RTS, using multi-spectral PlanetScope, derived topographic (ArcticDEM) and multi-temporal Landsat Trend data. We created a highly automated processing pipeline, which is designed to create reproducible results and to be flexible for multiple input features. The processing workflow is based on the pytorch deep-learning framework and includes a variety of different segmentation architectures (UNet, UNet++, DeepLabV3), backbones and includes common data transformation techniques such as augmentation or normalization. We tested (training, validation) our DL based model in six different regions of 100 to 300 km² size across Canada (Banks Island, Tuktoyaktuk, Horton, Herschel Is.), and Siberia (Kolguev, Lena). We performed a regional cross-validation (5 regions training, 1 region validation) to test the spatial robustness and transferability of the algorithm. Furthermore, we tested different architectures backbones and loss-function to identify the best performing and most robust parameter sets. For training the models we created a training database of manually digitized and validated RTS polygons. The resulting model performance varied strongly between different regions with maximum Intersection over Union (IoU) scores between 0.15 and 0.58. The strong regional variation emphasizes the need for sufficiently large training data, which is representative for the massive variety of RTS. However, the creation of good training data proved to be challenging due to the fuzzy definition and delineation of RTS, particularly on the lower part. We have recently expanded our analysis to several RTS-rich regions across the Arctic (Fig.X) for the year 2021 and annual analysis (2018-2021) for RTS hot-spots, e.g. Banks Island, Peel Plateau and others. First model inference runs are promising for detecting RTS, but are still strongly overestimating the number and area of RTS, due to an excessive number of false positives. Model performance however, varies strongly between regions. Due to the strong variability of landscapes with RTS, we expect an improvement in model performance with an increase in the number and spatial distribution of training datasets. The community driven formation of the IPA Action Group RTSIn, which aims to create standardized RTS digitization protocols and training datasets for deep/machine-learning purposes will be a great boost for our purpose. With our standardized processing pipeline (preprocessing, training, inference), which allows to add more features based on user interest and data availability,, we tested our workflow for surface water and pingos with a mixture of publically available (Jones et al) and digitized data (Grosse pingos, Nitze water). These tests produced very good results and showed that the designed workflow is transferrable beyond the segmentation of RTS only. In the near future, we are aiming to integrate the community based training data and further gradually improve our training database. Within the framework of the ML4Earth project, we will create a temporal and pan-arctic monitoring system for RTS based on our highly automated processing chain. This will enable us to better understand pan-arctic RTS dynamics, their influencing factors, and consequences. Combining these spatial-temporal datasets with volumetric change information and carbon stock information will enable us to better quantify the consequences of thaw slumping across the permafrost domain

HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline

Deep learning-based coastline detection algorithms have begun to outshine traditional statistical methods in recent years. However, they are usually trained only as single-purpose models to either segment land and water or delineate the coastline. In contrast to this, a human annotator will usually keep a mental map of both segmentation and delineation when performing manual coastline detection. To take into account this task duality, we therefore devise a new model to unite these two approaches in a deep learning model. By taking inspiration from the main building blocks of a semantic segmentation framework (UNet) and an edge detection framework (HED), both tasks are combined in a natural way. Training is made efficient by employing deep supervision on side predictions at multiple resolutions. Finally, a hierarchical attention mechanism is introduced to adaptively merge these multiscale predictions into the final model output. The advantages of this approach over other traditional and deep learning-based methods for coastline detection are demonstrated on a dataset of Sentinel-1 imagery covering parts of the Antarctic coast, where coastline detection is notoriously difficult. An implementation of our method is available at \url{https://github.com/khdlr/HED-UNet}.Comment: This work has been accepted by IEEE TGRS for publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Seeing the bigger picture: Enabling large context windows in neural networks by combining multiple zoom levels

When adopting deep learning methods for remote sensing applications, the data usually needs to be cut into patches due to hardware limitations. Clearly, this practice discards a lot of contextual information as the model's information is limited to imagery from the given patch. We propose a memory-efficient way around this limitation by using multiple patches of varying spatial extents on different resolution levels. Finally, this new approach is evaluated for the task of automated sea ice charting, where the added contextual information is shown to be beneficial to model performance

Deep learning based automatic grounding line delineation in DInSAR interferograms

The grounding line is a subsurface geophysical feature that divides the grounded ice sheet and floating ice shelf. Knowledge of its precise location is required for estimating ice sheet mass balance, as ice discharged from the interior is typically calculated at the grounding line [1], [2]. While grounding lines in Greenland have only a minimal extension, in Antarctica, they span about 75% of its coastline. The bending of ice shelves due to ocean tides causes them to migrate several kilometers over a tidal cycle within a transition region called the grounding zone. This short-term displacement adds to the difficulty in grounding line detection on a featureless ice surface. Nevertheless, various remote sensing methods can currently detect grounding lines on a continental scale. In particular, Differential Interferometric Synthetic Aperture Radar (DInSAR) is used to measure the deformation which occurs at the grounding line due to tidal flexure of ice shelves with sub-centimeter accuracy [3]. If coherence is preserved between the SAR repeat passes, the vertical ice deformation at the grounding zone is visible in the double difference interferogram as a dense fringe belt. The landward-most fringe is considered a good approximation of the actual grounding line. Although the generation of DInSAR interferograms is already automatized, the identification of the landward-most fringe and its digitization is still majorly performed manually by human operators. Besides being labour and time-intensive, manual delineations are inconsistent due to varying interpretations of experts in identifying the landward fringe, especially in areas with poor coherence or intricate fringe patterns. In the present study, we attempt to automate the delineation by employing a Convolutional Neural Network (CNN). We developed an automatic workflow that handles the preparation of the training feature stack, training and inference of the neural network and the post-processing of network-generated delineations. The CNN architecture is based on the Holistically-Nested Edge Detection network [4]. It was trained on 478 georeferenced DInSAR interferograms from ERS-1/2, Sentinel-1 A/B and TerraSAR-X repeat pass acquisitions and their corresponding hand-delineated grounding lines that were generated within the Grounding Line Location (GLL) product of ESA’s Climate Change Initiative (AIS cci) project [5]. The training feature stack consists of four interferogram-based features: real and imaginary components, interferometric phase and pseudo coherence (estimated by applying a boxcar filter to interferometric phase) derived from the corresponding DInSAR interferograms. A median deviation of 209 m between the network-delineated and corresponding manual GLLs was measured for the test set. The trained network delineates an interferogram in milliseconds, considerably shorter than the time required for manual delineation. We propose to automatically and efficiently expand the AIS cci GLL product by applying our trained neural network to interferograms that still need to be manually delineated. In particular, we plan to generate DInSAR interferograms from highly coherent TerraSAR-X data triplets acquired in 2021 using the Integrated Wide Area Processor (IWAP) [6]. These acquisitions were made over Southern Byrd, Amundsen, Lennox-King and Dickey glaciers feeding into the Ross Ice Shelf and Recovery Glacier situated in the Ronne-Filchner Ice Shelf at high latitudes, which Sentinel-1 cannot image. Consequently, no updated grounding lines for these glaciers exist in current DInSAR-based grounding line datasets [7]. In general, the performance of our trained neural network is not dependent on the SAR sensor but on the quality of the interferograms. The automatic delineation can create monthly or half-yearly average GLL time series from all suitable DInSAR interferograms in a certain period. This derived product has a downstream application in analyzing short and long-term migratory patterns of grounding lines. References [1] C. Schoof, “Ice sheet grounding line dynamics: Steady states, stability, and hysteresis,” Journal of Geophysical Research: Earth Surface, vol. 112, no. F3, 2007. [2] E. Rignot and H. Thomas, “Mass balance of polar ice sheets,” Science, vol. 297, no. 5586, pp. 1502–1506, 2002. DOI: 10.1126/science.1073888. eprint: https://www.science.org/doi/pdf/10.1126/science.1073888. [Online]. Available: https://www.science.org/doi/abs/10.1126/science.1073888. [3] E. Rignot, “Tidal motion, ice velocity and melt rate of petermann gletscher, greenland, measured from radar interferometry,” Journal of Glaciology, vol. 42, no. 142, pp. 476–485, 1996. [4] S. Xie and Z. Tu, “Holistically-nested edge detection,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1395–1403. [5] A. Groh, Product user guide (pug) for the antarctic ice sheet cci project of esa’s climate change initiative, version 1.0, 2021. [Online]. Available: https://climate.esa.int/media/documents/ST- UL- ESA- AISCCI- PUG- 0001.pdf. [6] F. R. Gonzalez, N. Adam, A. Parizzi, and R. Brcic, “The integrated wide area processor (iwap): A processor for wide area persistent scatterer interferometry,” in ESA Living Planet Symposium, vol. 722, 2013, p. 353. [7] E. Rignot, J. Mouginot, and B. Scheuchl, “Measures antarctic grounding line from differential satellite radar interferometry, version 2,” NASA, 2016. [Online]. Available: https://doi.org/10.5067/IKBWW4RYHF1Q

Mapping the grounding line of Antarctica in SAR interferograms with machine learning techniques

The grounding line marks the transition between ice grounded at the bedrock and the floating ice shelf. Its location is required for estimating ice sheet mass balance [Rignot & Thomas, 2002], modelling of ice sheet dynamics and glaciers [Schoof 2007], [Vieli & Payne, 2005] and evaluating ice shelf stability [Thomas et al., 2004], which merits its long-term monitoring. The line migrates both due to short term influences such as ocean tides and atmospheric pressure, and long-term effects such as changes of ice thickness, slope of bedrock and variations in sea level [Adhikari et al., 2014]. The grounding line is one of four parameters characterizing the Antarctic Ice Sheet (AIS) ECV project within ESA’s Climate Change Initiative (CCI) programme. The grounding line location (GLL) geophysical product was designed within AIS_CCI and has been derived through the double difference InSAR technique from ERS-1/2 SAR, TerraSAR-X and Sentinel-1 data over major ice streams and outlet glaciers around Antarctica. In the current stage of the CCI project, we have interferometrically processed dense time series throughout the year from the Sentinel-1 A/B constellation aiming at monitoring the short-term migration of the DInSAR fringe belt with respect to different tidal and atmospheric conditions. Whereas the processing chain runs automatically from data download to interferogram generation, the grounding line is manually digitized on the double difference interferograms. Inconsistencies are introduced due to varying interpretation among operators and the task becomes more challenging when using low coherence interferograms. On a large scale this final stage of processing is time consuming, hence urging the need for automation. The grounding line is one of four parameters characterizing the Antarctic Ice Sheet (AIS) ECV project within ESA’s Climate Change Initiative (CCI) programme. The grounding line location (GLL) geophysical product was designed within AIS_CCI and has been derived through the double difference InSAR technique from ERS-1/2 SAR, TerraSAR-X and Sentinel-1 data over major ice streams and outlet glaciers around Antarctica. In the current stage of the CCI project, we have interferometrically processed dense time series throughout the year from the Sentinel-1 A/B constellation aiming at monitoring the short-term migration of the DInSAR fringe belt with respect to different tidal and atmospheric conditions. Whereas the processing chain runs automatically from data download to interferogram generation, the grounding line is manually digitized on the double difference interferograms. Inconsistencies are introduced due to varying interpretation among operators and the task becomes more challenging when using low coherence interferograms. On a large scale this final stage of processing is time consuming, hence urging the need for automation. This study further investigates the feasibility of automating the grounding line digitization process using machine learning. The training data consists of double difference interferograms and corresponding manually delineated AIS_CCI GLL’s derived from SAR acquisitions between 1996 - 2020 over Antarctica. In addition to these, features such as ice velocity, elevation information, tidal displacement, noise estimates from phase and atmospheric pressure are analyzed as potential inputs to the machine learning network. The delineation is modelled both as a semantic segmentation problem, as well as a boundary detection problem, exploring popular existing architectures such as U-Net [Ronneberger et al., 2015], SegNet [Badrinarayanan et al., 2017] and Holistically-nested Edge Detection [Xie & Tu, 2015]. The resulting grounding line predictions will be examined with respect to their usability in the detection of short-term variations of the grounding line as well as the potential separation of a signal of long-term migration. The detection accuracy will be compared to the one achieved by human interpreters. Adhikari, S., Ivins, E. R., Larour, E., Seroussi, H., Morlighem, M., and Nowicki, S. (2014). Future Antarctic bed topography and its implications for ice sheet dynamics, Solid Earth, 5, 569–584 Baumhoer, C. A., Dietz, A. J., Kneisel, C., & Kuenzer, C. (2019). Automated extraction of antarctic glacier and ice shelf fronts from sentinel-1 imagery using deep learning. Remote Sensing, 11(21), 2529 Badrinarayanan, V., Kendall, A., Cipolla, R., (2017). Segnet: A deep convolutional encoder-decoder architecture for scene segmentation. IEEE transactions on pattern analysis and machine intelligence. Cheng, D., Hayes, W., Larour, E., Mohajerani, Y., Wood, M., Velicogna, I., & Rignot, E. (2021). Calving Front Machine (CALFIN): glacial termini dataset and automated deep learning extraction method for Greenland, 1972–2019. The Cryosphere, 15(3), 1663-1675 Krieger, L., & Floricioiu, D. (2017). Automatic calving front delienation on TerraSAR-X and Sentinel-1 SAR imagery. In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) Mohajerani, Y., Jeong, S., Scheuchl, B., Velicogna, I., Rignot, E., & Milillo, P. (2021). Automatic delineation of glacier grounding lines in differential interferometric synthetic-aperture radar data using deep learning. Scientific reports, 11(1), 1-10. Rignot, E., & Thomas, R. H. (2002). Mass balance of polar ice sheets. Science, 297(5586), 1502-1506. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Springer International Publishing: Cham, Switzerland, 2015; Volume 9351, pp. 234–241. ISBN 978-3-319-24573-7 Schoof, C. (2007). Ice sheet grounding line dynamics: Steady states, stability, and hysteresis, J. Geophys. Res., 112, F03S28, doi:10.1029/2006JF000664. Xie, S., Tu, Z., 2015. Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1395-1403 Thomas, R., Rignot, E., Casassa, G., Kanagaratnam, P., Acuña, C., Akins, Brecher, H., Frederick, E., Gogineni, P., Krabill, W., Manizade, S., Ramamoorthy, H., Rivera, A., Russell, R., Sonntag, J., Swift, R., Yungel, J., & Zwally, J., (2004). Accelerated sea-level rise from West Antarctica. Science, 306(5694), 255-258. Vieli, A., & Payne, A. J. (2005). Assessing the ability of numerical ice sheet models to simulate grounding line migration, J. Geophys. Res., 110, F01003, doi:10.1029/2004JF00020

Deep neural network based automatic grounding line delineation in DInSAR interferograms

The grounding line is a subsurface geophysical feature that divides a grounded ice sheet and floating ice shelf. Knowledge of its precise location is required for estimating ice sheet mass balance, as ice discharged from the interior is typically calculated at the grounding line [1]. Grounding lines move back and forth as ice shelves bend and flex due to ocean tides. Identifying their migration patterns can provide insights into understanding ice sheet dynamics and overall ice sheet stability [2] and thereby improve the accuracy of numerical ice sheet models. The spatial and temporal resolution of past and current satellite missions has enabled regular, continent-wide observation of Antarctica and other isolated glaciers with floating ice tongues. In particular, the high sensitivity of Interferometric SAR measurements to ground deformation has resulted in its application to grounding line location (GLL) mapping [3]. Specifically, the deformation at the grounding zone resulting from tidal flexure of the ice shelf is isolated from ice motion and topography in Differential InSAR (DInSAR) interferograms, under the assumption of steady ice velocity within the chosen temporal baseline. The tidal deformation is visible as a dense fringe belt and its landward limit is manually digitised as the GLL. Apart from being labour and time intensive, manual delineations are also inconsistent due to varying interpretations of experts in identifying the landward fringe, especially in areas with poor coherence or intricate fringe patterns. The concept of automatic GLL delineation has recently gained attention and seen the development of several methodologies. [4] demonstrated a semi-automatic method that estimates the fringe frequency of wrapped phase in DInSAR interferograms. The grounding zone can be directly identified by computing the gradient of the estimated frequencies, thereby avoiding phase unwrapping. However, this approach requires an a priori grounding zone location. [5] developed a deep learning based automatic delineation pipeline in which the proposed DNN was trained on real and imaginary components of DInSAR phases from Sentinel-1 acquisitions. This study further investigated the feasibility of DNNs for mapping the interferometric grounding line. The proposed DNN, based on the architecture of the Holistically-Nested Edge Detection network [6], was trained in a supervised manner, using manual delineations from the GLL product developed within ESA’s Antarctic Ice Sheet climate change initiative (AIS cci) project [7] as ground truth (Fig. 1 (a)). The GLL product contains manual delineations on 478 DInSAR interferograms computed from Sentinel-1A/B, ERS-1/2 and TerraSAR-X images acquired during 1992 - 2021. The training feature stack consists of four interferogram-based features: real and imaginary components, interferometric phase and pseudo coherence (which is estimated by applying a boxcar filter to interferometric phase) derived from the corresponding DInSAR interferograms and five auxiliary features derived from several compiled datasets: TanDEM-X Polar DEM [8], horizontal and vertical components of ice velocity [9], tidal amplitude [10] and atmospheric pressure [11]. An automatic workflow that handles the preparation of the training feature stack, training and inference of the neural network and the post processing of network generated delineations was developed. The performance of the neural network was evaluated as the median deviation of the network generated GLLs from the manual delineations, quantified using the PoLiS metric [12]. Additionally, the importance of individual features was indirectly gauged by training several networks with different feature subsets and comparing their median deviations from the ground truth. The DNN generated GLLs follow the landward limit of ice sheet flexure reasonably well, with the best network variant achieving a median deviation of 209 m from manual delineations.The contribution of auxiliary features was shown to be very weak, with their inclusion in the feature stack only slightly improving the delineation capability of the network. This finding is advantageous in terms of saving time, computational effort and memory in creating and storing the feature stack. References [1] E. Rignot and H. Thomas, “Mass balance of polar ice sheets,” Science, vol. 297, no. 5586, pp. 1502–1506, 2002. DOI: 10 . 1126 / science . 1073888. eprint: https : / / www . science . org / doi / pdf / 10 . 1126 / science.1073888. [Online]. Available: https://www.science.org/doi/abs/10.1126/science.1073888. [2] C. Schoof, “Ice sheet grounding line dynamics: Steady states, stability, and hysteresis,” Journal of Geophysical Research: Earth Surface, vol. 112, no. F3, 2007. [3] E. Rignot, “Tidal motion, ice velocity and melt rate of petermann gletscher, greenland, measured from radar interferometry,” Journal of Glaciology, vol. 42, no. 142, pp. 476–485, 1996. [4] A. Parizzi, “Potential of an Automatic Grounding Zone Characterization Using Wrapped InSAR Phase,” in IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA: IEEE, Sep. 2020, pp. 802–805, ISBN: 978-1-72816-374-1. DOI: 10.1109/IGARSS39084.2020.9323199. [5] Y. Mohajerani, S. Jeong, B. Scheuchl, I. Velicogna, E. Rignot, and P. Milillo, “Automatic delineation of glacier grounding lines in differential interferometric synthetic-aperture radar data using deep learning,” Scientific reports, vol. 11, no. 1, pp. 1–10, 2021. [6] S. Xie and Z. Tu, “Holistically-nested edge detection,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1395–1403. [7] A. Groh, Product user guide (pug) for the antarctic ice sheet cci project of esa’s climate change initiative, version 1.0, 2021. [Online]. Available: https://climate.esa.int/media/documents/ST-UL-ESA-AISCCI-PUG-0001.pdf. [8] M. Huber, Tandem-x polardem product description, prepared by german remote sensing data center (dfd) and earth observation center, 2020. [Online]. Available: https://www.dlr.de/eoc/en/desktopdefault.aspx/tabid11882/20871_read-66374. [9] T. Nagler, H. Rott, M. Hetzenecker, J. Wuite, and P. Potin, “The sentinel-1 mission: New opportunities for ice sheet observations,” Remote Sensing, vol. 7, no. 7, pp. 9371–9389, 2015. [10] L. Padman, S. Erofeeva, and H. Fricker, “Improving antarctic tide models by assimilation of icesat laser altimetry over ice shelves,” Geophysical Research Letters, vol. 35, no. 22, 2008. [11] E. Kalnay, M. Kanamitsu, R. Kistler, W. Collins, D. Deaven, L. Gandin, M. Iredell, S. Saha, G. White, J. Woollen, et al., “The ncep/ncar 40-year reanalysis project,” Bulletin of the American meteorological Society, vol. 77, no. 3, pp. 437–472, 1996. [12] J. Avbelj, R. Muller, and R. Bamler, “A metric for polygon comparison and building extraction evaluation,” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 1, pp. 170–174, 2014

HED-UNet: A multi-scale framework for simultaneous segmentation and edge detection

Segmentation models for remote sensing imagery are usually trained on the segmentation task alone. However, for many applications, the class boundaries carry semantic value. To account for this, we propose a new approach that unites both tasks within a single deep learning model. The proposed network architecture follows the successful encoder-decoder approach, and is improved by employing deep supervision at multiple resolution levels, as well as merging these resolution levels into a final prediction using a hierarchical attention mechanism. This framework is trained to detect the coastline in Sentinel-1 images of the Antarctic coastline. Its performance is then compared to conventional single-task approaches, and shown to outperform these methods. The code is available at https://github.com/khdlr/HED-UNe

The Potential of Artificial Intelligence and Remote Sensing for Cryospheric Research

Recent advances in artificial intelligence, especially in the field of deep learning, have allowed new insights into cryospheric systems. Nowadays, an abundance of satellite imagery, new developments in deep learning algorithms and easy accessibility to computational power enable new potentials for data processing and analysis. Here, we present a variety of deep learning applications for cold and polar regions providing new possibilities for observing and monitoring the cryosphere. The presented examples cover a wide range of applications such as mapping retrogressive thaw slumps in Arctic permafrost regions with high-resolution satellite imagery based on a UNet++ or the automated identification of the firn line in L-Band SAR data. Furthermore, methodologies for glacial lake mapping in the Himalayas with the GLNet and the detection of supraglacial lake dynamics in Antarctica based on optical and SAR satellite data will be introduced. Additionally, we address the automated extraction of calving fronts in Greenland and Antarctica providing new understandings of glacier and ice shelf front dynamics in an unprecedented spatial and temporal resolution. Taking together these new potentials of artificial intelligence for cold and polar regions, we welcome discussions on how these techniques can be applied to other areas in cryospheric science and what challenges and limitations this might involve